• Lots of files
• Cut and paste is not a reliable, reproducible approach!
• Each operator has his/her own approach
• Several versions of code may coexist
• Mistakes hard to track
• The steps aren’t recorded
• Testing is hard
• Reproducibility is not granted
• Quality is controlled only at the end

• It is a process
• It is easily repeatable
• It is easily extendable
• It is automated
• It minimises mistakes
• It is fast
• It builds trust

It is a simple process:

• linking inputs (data)
• to outputs (publication)

This process can be decomposed:

• Succession of tasks
• Direct linkage of actions
• Different software can be used

This process can be decomposed:

• Each task is coded
• \(\hookrightarrow\) No manual actions
• Each task uses inputs
• Each task produces outputs
• \(\hookrightarrow\) Easy to test tasks individually
• \(\hookrightarrow\) Each output is identified

This process is documented:

• Each code has versions
• Versions are annotated
• \(\hookrightarrow\) Easy to follow tasks development
• \(\hookrightarrow\) Easy to track mistakes

This process is saved:

• Each code is securely saved
• Each version can be revereted
• \(\hookrightarrow\) Easy to undo/revert to past version
• \(\hookrightarrow\) Easy to test

Analysis within an RAP are:

• Easy to use
• Easy to find information
• Easy for others to use
• Easy to revise and adapt
• Easy to reuse
• Automated and fast
• Open and promoting trust

• A good knowledge of the process

• A good organisation:

  • of files
  • of code
  • of documentation

• An open source software

• A versioning system
  

• Time to learn

Open source tools:

• Have a huge supportive online community
• Are reviewed. Issues fixed swiftly
• Are transparent in their content
• Can inter operate with other software
• Are free for anyone to use and share

Tracking the three Ws:

• Who made this?
• Which change(s)?
• Why?

• One place to store your code
• You and collaborators are free to write and develop locally
• Complete documented history of all changes made
• Easy to share
• Your future self will thank you!

An analysis can be:

• Reproducible
• Replicable
• Robust
• Reusable

A project is reproducible if it returns the same results when redone with the same data and the same analysis (same code).

What are the benefits?

• Helps build trust
• Not reliant on single individual
• Can be adapted and re-used

Before we start, here are a few things to consider:

• IT infrastructure available
• Data privacy - where and how am I storing my data?
• Expertise - what training do I need?
• Legacy systems - what are the barriers to transitioning?

The building blocks of a RAP:

• Reproducible code
• Using open-source tools
• Version control
• … all useful, each will improve a specific dimension of the process.

• Implemented in some NSOs (Vanuatu)
• Can be done easily with R/Rstudio
• Can also be done with Python/Jupyter notebooks, Quarto (both R, Python, Julia, others…)
• Large community to help

• The UK government RAP website.

• UK best practice documentation.

• A free RAP course to teach you all you need to know.

• How the Data Science Campus sets its coding standards.

• A new open-source book from the Alan Turing institute setting out how to do reproducible data science.

Many of the beautiful images used in this presentation were taken from The Turing Way book.

Full citation:

The Turing Way Community, Becky Arnold, Louise Bowler, Sarah Gibson, Patricia Herterich, Rosie Higman, … Kirstie Whitaker. (2019, March 25). The Turing Way: A Handbook for Reproducible Data Science (Version v0.0.4). Zenodo. http://doi.org/10.5281/zenodo.3233986